An Analysis of 

Space Shuttle Countdown Activities: 
Preliminaries to a Computational Model 
of the NASA Test Director. 

Bonnie E. John 1 Roger W.Remington 2 David M. Steier 3 

31 May 1991 
CMU-CS-91-138 


School of Computer Science 
Carnegie Mellon University 
Pittsburgh, PA 15213-3890 


Abstract 

Before we hear the familiar phrase "All systems are go" just prior to the launch of a space 
shuttle, thousands of operations and tests have been performed to ensure that all shutde 
and support subsystems are operational and ready for launch. These steps, which range 
from activating the orbitetis flight computers to removing the launch pad from the itinerary 
of the NASA tour buses, are carried out by launch team members at various locations and 
with highly specialized fields of expertise. The responsibility for coordinating these 
diverse activities rests with the NASA Test Director (NTD) at Kennedy Space Center. We 
are studying the behavior of the NTD with the goal of building a detailed computational 
model of that behavior, this paper describes the results of our analysis to date. We describe 
the NTD's performance in detail, as a team member who must coordinate a complex task 
through efficient audio communication, as well as an individual taking notes and consulting 
manuals. A model of the routine cognitive skills employed by the NTT) to follow the 
launch countdown procedure manual has been implemented using the Soar cognitive 
architecture. The paper concludes with several examples of how such a model could aid in 
evaluating proposed computer support systems. 
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1. INTRODUCTION 

We are studying the job of the NASA Test Director (NTD) at the Launch Control Center 
(LCC) at Kennedy Space Center (KSC) with two objectives. The first is to build a 
computer simulation model of the perceptual, cognitive, and motor processes performed by 
the NTD in the course of preparing for launch; the second is to use that model to evaluate 
and indicate changes in the NTD job, training, or computer support aids, that would 
facilitate or improve performance during launch preparation. 

The NTD is one member of a team of hundreds of engineers and technicians, the launch 
team, that performs the many tests and verification procedures leading up to the launch of a 
space shuttle. The NTD, in particular, is responsible for coordinating and tracking the 
progress of those procedures. In addition, the NTD is responsible for managing any 
emergency actions necessary during a launch countdown. This job, especially late in the 
countdown or under emergency conditions, is extremely demanding. In the course of a 
typical countdown, the NTD schedules thousands of crucial tests and verification 
procedures for all shuttle systems, processes hundreds of requests from other team 
members for the limited resources of time, space, and equipment, and monitors tens of 
communications channels simultaneously. 

Shuttle missions are more frequent than previous manned space programs, and it is the 
objective of the shuttle program to increase frequency even further. An increase in launch 
frequency raises concerns about greater efficiency and decreased NTD training time. One 
path to greater efficiency involves a major hardware and software upgrade now in progress 
that will provide the opportunity for substantial improvements in display and control 
technology. Given this opportunity to develop new computer support systems, NASA 
seeks to understand the knowledge and processing demands of the NTD's job during 
launch. Such an understanding may allow the evaluation of computer support systems md 
their impact on both expert performance and training. Our study of the job of the NTD is a 
first step toward understanding, evaluating, and modifying those demands. 

Our work begins with field observations of the NTD's job and analyses of the behavior 
observed. We then proceed to model mechanisms sufficient to produce that behavior 
within the Soar cognitive architecture (Newell, 1990). Our goal is to produce a full-fledged 
psychological model that produces the same behavior as an expert NTD, with an indication 
of how much time that behavior will take and what knowledge structure and situational 
information is necessary, within human perceptual, motor, and information processing 
capabilities. We expect that this research will also contribute more generally to the 
development of a computational framework for analyzing other complex tasks. 

We expect that a mechanistic, computer- simulated psychological model of NTD 
performance will provide information relevant to the design of the NTD's job and training. 
The model itself will provide a testbed for design changes. For instance, if a new visual 
display of status information is suggested, the features of this display could be given to the 
model with different task scenarios and the model would provide a prediction of the 
changes in performance due to that display. In addition, the model will provide a trace of 
the internal processes involved in performance. This trace can give insight into what 
changes might best benefit the NTD. For example, the model may indicate that at a 
particular point in the job, much effort is used switching attention between several tasks not 
because they need to be monitored, but because constant refreshing of the model's memory 
is required to be ready in case a task turns critical. This might suggest that an external 
display of the critical information could be introduced to relieve this memory and attention 
burden. In terms of training, the model will provide information about the possible 
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knowledge representation used by NTDs. Analysis of the use of these structures may 
suggest a change in training that would produce more efficient representations. 

2. THE JOB OF THE NASA TEST DIRECTOR 

The launch of a space shuttle is the culmination of a sequence of tests and construction 
involving several hundred people over the course of several months. The solid rocket 
booster, main engine, payload, and the orbiter itself are tested and outfitted separately by 
teams of specialists, then mated and moved as a unit onto the launch pad. Testing and 
preparation continue with the vehicle on the launch pad until all critical systems, including 
those associated with the launch pad and the LCC itself, are verified to be functioning 
within normal limits. From the time a shuttle orbiter returns from its mission until it is 
launched again over 100,000 staff-hours of work will have been done involving more then 
2000 people in performing thousands of separate tests. 

The NTD is that member of the launch team responsible for coordinating all phases of 
shuttle preparation; he is the focal point for all engineering and safety tests conducted 
during countdown. The NTD must manage and implement the schedule that guides shuttle 
progress during all phases of preparation. During die final phases of countdown, the NTD 
must make critical, time-constrained decisions concerning launch status, while fielding a 
constant stream of auditory messages from other team members that update that status. It is 
important to emphasize that although technical problems are reported to the NTD, the NTD 
is not a technical troubleshooter. Rather, the NTD coordinates launch activities, including 
directing communication between technical troubleshooters, that is, the NTD must know 
whom to ask if there is trouble. Ultimately, the NTD must give the Launch Director the 
final assurance that the shuttle is ready to launch. 

The NTD is physically located in the Firing Room of the LCC. Figure 1 shows some of 
the stations and their approximate layout in the Firing Room. During countdown the Firing 
Room is staffed by a team of over 200 engineers and technicians at stations concerned with 
specific launch subsystems, including fuel, engines, software, electrical systems, and 
instrumentation. The NTD position is in a row elevated slightly above the subsystem 
workstations. During launch, eight or more people may occupy the NTD row. These 
people include the Prime NTD and the Assistant NTD, who shares some tasks with the 
Prime NTD. (Our observations indicate that the Prime NTD remains at the NTD station, 
performs all the communication, and much of the note-taking, during the time period of 
interest, T-20 min to launch. Our use of the term "NTD" refers to the Prime NTD.) The 
NTD can see and communicate directly with the people on his row, and by standing up can 
see over his equipment to the some, but not all, of the other workstation operators. Above 
and behind the NTD is the Launch Director who is responsible for the final OK for launch. 

The primary form of communication between the NTD and other members of the launch 
team is over a two-way radio net, the Operational Intercommunications System (OIS). 

The NTD has one OIS channel assigned to him, but a dozen communi catio ns channels 
appear on his console. In addition to continually monitoring his own channel, the NTD 
usually monitors several of the other channels simultaneously at reduced volume, all mixed 
binaurally. Throughout most of the countdown, the NTD may also initiate or receive calls 
on a private telephone line or make announcements on a public address system. In the last 
20 minutes of the cou ntdown, al l communications between members of the launch team 
switch to a single OIS channel, Channel 212. •• - 


The primary source of visual information for the NTD is the volumes of manuals that 
contain the steps necessary to launch the shuttle, collectively known as the Operations and 
Maintenance Instructions (OMI). The OMI has over 3000 pages, including normal, abort 
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(or scrub), and emergency procedures divided into about 50 sequences. Each sequence 
contains a number of individual steps of varying duration and complexity to be performed 
during countdown. For example, the countdown from T minus 1 1 hours to T minus 6 
hours, the pad closeout phase which the NTDs regard as their busiest time, is covered by 
the 622 steps of Sequence 15, while the time from T minus 6 hours to T-0 falls under the 
1132 steps of Sequence 16. Each entry in die OMI (example page in Figure 2) lists a test 
or procedure that must be performed to verify a subsystem (DESCRIPTION column), the 
approximate time at which this step should be completed (TIME column), and the 
communications protocol that must be followed to complete the step (CMD and RESP 
columns). The steps are listed sequentially, and given a unique step number (the SEQ 
column), but in fact many procedures are done in parallel. Constraints on the order of 
procedure execution are sometimes specified. Not all the steps in the OMI will be 
performed during any one launch; while some are common to all launches, others occur 
only in the event of an abort or recycle. Some steps depend on factors such as the duration 
of the flight, the weather, and the type of payload. Other documents consulted by the NTD 
are detailed bar chart schedules and a listing of acceptable ranges for thousands of 
parameters of shuttle systems (the Launch Commit Criteria). 

The NTD works from an integrated OMI (which we will call, simply, the OMI). Each 
subsystem manager has a separate subsystem OMI that gives detailed procedures for 
carrying out the tests specified in the OMI. Unless the procedure specified in the subsystem 
manual is dependent on the completion of a procedure in another subsystem, or is 
dependent on some set of initial conditions, subsystem managers will initiate tests without 
clearance from the NTD. The OMI explicitly specifies when such clearance is needed. 

The OMI has evolved from the collective experience of launch teams over the years of 
NASA's space exploration. In the days of the Mercury project, launch teams developed a 
list of procedures in order to avoid omissions and to insure that the launch sequence was 
carried out in the correct order. To help avoid these problems in subsequent launches, they 
prepared a "guide to launch", listing the major tests and their sequence. Around the time of 
the first shuttle launch (circa 1980) the initial boilerplate for the current OMI was 
developed, along with a manual. The Technical Operating Procedures (TOPS) Preparation 
Handbook , that delineated rules and procedures for constructing and modifying the OMI. 

Changes to the OMI can be proposed by any member of the launch team when he or she 
discovers an error or thinks of a way to improve performance. The formal process for 
proposing and processing changes is described in the TOPS Preparation Handbook. Each 
proposal is discussal by an appropriate subset of the launch team and a written deviation is 
prepared. Every deviation is documented; nothing changes, nor is any part of any task 
begun, until it is written down in the appropriate document according to procedures set 
forth in the TOPS Preparation Handbook. Some changes require approval at the NTD level; 
others require higher levels of approval. In any case, no change is the result of a single 
person’s judgment, but involves the appropriate team members. As would be expected in 
such an evolutionary process, many more deviations were processed in the early years of 
shuttle launches than are now processed. This revision process, involving the team in 
proposal and discussion phases, and building on previous launches over a period of years, 
results in an OMI that is highly trusted and adhered to, but is viewed as a document that can 
be changed when the need arises. 


The current set of visual inputs normally used by the NTD does not include any integrated 
computer display on which the progress of the launch can be viewed. Several small CRT 
screens appear at the NTD's console but are used to display closed circuit television 
pictures of the launch pad and related facilities, rather than the output of any computer 
support tools. There are in fact, many computer consoles in the firing room, but it is the 
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responsibility of the personnel at the subsystem and integration stations where these 
consoles are located to interpret the displays, and relay any significant developments to 
management over the OIS communication network. 

When a subsystem has completed the tests required to fulfill an OMI procedure, the 
subsystem manager calls the NTD on an OIS channel and informs him of the step 
completion. In practice, such completions are often grouped, so that the NTD receives a 
sequence of step completions from a subsystem manager. Upon receiving a step 
completion, the NTD marks the step as completed in his copy of the OMI. Again, in 
practice, the NTD may acknowledge the completion, write it on a piece of note paper, and 
defer the sign-off in the OMI until later. If steps are skipped or not yet completed, the NTD 
may insert a Post-It ™ in the OMI to mark the uncompleted page. In addition, the NTD 
makes log entries that serve as a record of activity. Log book notations are especially 
important when anomalies occur. 

If a system fails, tests fall outside their normal ranges, or any of a number of anomaly 
conditions hold, the NTD coordinates the response. A problem may involve a number of 
subsystems; the NTD may establish a separate communications channel and assign 
personnel from the appropriate subsystems to communicate technical information over that 
channel. The NTD is also responsible for predicting the effect of the anomaly on other 
aspects of the launch. A failure in one subsystem may cause procedures in other 
subsystems to be rescheduled. The countdown clock can be stopped, and there are fixed 
holds which can be used to accommodate such delays. At other times, as, for example, 
once the hydrogen and liquid oxygen have been loaded, the clock can be held only a short, 
fixed, time before the launch must be scrubbed. Dealing with anomalies is a difficult aspect 
of the NTD's job. The launch is dynamic, and conditions (such as weather) change 
quickly, so that holds must be carefully planned. Response to an anomaly involves 
increased planning, increased communications, and critical decision-making. 

As previously mentioned, there is little or no computer support used routinely by the NTD 
to aid in the various tasks. Although the OMI itself is updated often and maintained 
electronically to facilitate those changes, its on-line version is not accessible to the NTD. 
Bookkeeping activities, such as marking pages in the OMI or making notes about status, 
are done by hand. There is no integrated display on which the progress of the launch can be 
viewed. Rather, this information is primarily kept in the NTD's head. There are few 
external memory aids that can be easily accessed — the OMI is the principal source — and 
no on-line aids to track the progress of on-going procedures. 

The job of the NTD has evolved to fit the methods and facilities available. The possible 
introduction of new facilities with the major hardware and software upgrade now in 
progress may suggest changes to these methods. It is our goal that this research will aid in 
evaluating such changes so that a technical revolution in information capabilities will not 
undermine the evolution in cooperative work that has produced the current standard of 
efficiency and safety. 

3. DATA COLLECTION AND DESCRIPTIVE ANALYSIS 

We spent a week in the LCC at KSC to become familiar enough with the NTD's job to 
begin analysis. The NTD's office graciously made several sources of information 
available to us: direct observation of NTDs performing both routine and critical operations, 
conversations with NTDs when they were not on duty, copies of the OMI and other 
documentation, audio tapes of actual pre-launch communications, and discussions with a 
member of the training team who conducts simulations of countdowns from T-20 min to 
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launch or scrub. (Simulations are used to train apprentice N'iDs in launch procedures and 
keep experienced NTDs familiar with rarely used emergency procedures.) At this p oint in 
the research, we were not able to observe the portions of the count down when the NTD is 
busiest (T-l 1 hours to T-6 hours), nor could we observe a launch. 

Based on the observations and conversations, we selected T-20 min to launch as the first 
segment of the countdown to study. This period is busy enough to be interesting and 
challenging, as our descriptive analysis will show, and yet structured enough to allow us to 
begin with a limited model. For example, rather than modeling the NTD as he monitors 
several communications channels simultaneously, our model can be limited to a single 
auditory input channel because all launch-critical communications are on channel 212 
during this period. Also, this segment is perceived to be more routine, containing fewer 
deviations from prescribed order of steps, and to be more similar from launch to launch by 
the NTDs. An additional benefit is that the T-20 min to launch segment of the countdown is 
the segment that can be simulated in training runs; when our model makes predictions about 
performance with different computer-support aids, we may be able to test these predictions 
via simulation. 

The NTD's office gave us audio tapes of the NTD's communication channel from three 
actual launch attempts: two launch attempts on consecutive days that were scrubbed and a 
third that ended in a successful launch two days later. We transcribed and timestamped the 
first attempt from T-20 min to scrub at T-31 sec, the second attempt from T-20 to scrub 
after an extended hold at T-9 min, and the third attempt from T-20 min to launch. Earlier 
portions of these launch attempts were listened to but not analyzed in detail. In addition, we 
were given an audio tape of the NTD's channel from a training simulation. At our request, 
the NTD's office videotaped activity from T-l 1 hours to T-6 hours. Although this is not the 
launch phase we will initially model, the NTDs view this time period as their busiest and 
most critical time. Portions of the video tapes were viewed and the physical interactions 
with equipment and personnel were noted. In general, the same types of interactions occur 
here as in the T-20 to launch time period, however the frequency of activities may differ. 

Before beginning a quantitative description of aspects of the NTD's performance, it is 
useful to characterize NTD behavior qualitatively. There is much cyclic, quasi-predictable 
sequenced behavior. During quiet periods, the NTD is either awaiting status update 
information, or seeking launch status information. When a subsystem manager calls the 
NTD to report status information an activity sequence begins that seems to be a local, 
immediate response to that status information. The NTD acknowledges the message, finds 
the indicated step in the OMI manual, and updates the manual. This patterned activity may 
vary in slight detail from one NTD to another, and during periods of high message load, 
the NTD may write notes indicating which steps have been completed rather than searching 
the manual at the time the step is given. The occurrence of unexpected events - failure 
conditions, or adverse weather - disrupts the programmed flow of events. The NTD must 
then track and amend the progress of the launch concurrently with the progress of the 
anomalous conditions. 

To illustrate the NTD’s behavior we created a variant of problem behavior graphs (Newell 
and Simon, 1972) that capture the nominal flow of events as described in the OMI, and the 
actual sequence of events, including interruptions. Figure 3 is our analysis of a simulated 
launch attempt; Figure 4 of an actual launch attempt. The ovals represent phrases spoken in 
a single message by a single individual. The darkest grey ovals are messages spoken by the 
NTD, the lighter grey ovals are addressed to the NTD, and the white ovals are messages on 
channel 212 that do not directly involve the NTD. The column labelled "Prescribed 
Procedure" contains a column of ovals whose contents are the step numbers in the OMI and 
numbers associated with the messages about those steps. This column is ordered in the 
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sequence in which the steps occur in the OMI. The arrows to the left of the ovals connect 
steps in the order in which they occurred in the transcript For example, the sequence of 
events starting with step 90 in Figure 4 went fro step 090 to step 106 to step 107 to step 
101. The dotted lines surrounding the ovals in the Prescribed Procedure column indicate 
which procedures are on the same page; crossing a dotted lines means the NTD had to flip a 
page to see that procedure. To the right of the Prescribed Procedure column are 
interruptions. These are messages which are not directly related to sequence steps, but refer 
to anomaly reports or other status information that require a response. The numbers do not 
correspond to steps in the OMI, but are a running count of the messages dealing with 
interruptions. 

The graph of the simulated launch, Figure 3, shows a relatively regular following of the 
prescribed procedures, with long interruptions due to anomalies inserted into the 
simulation. This is to be expected because the purpose of t hes e simulations is to familiarize 
the NTDs with rarely used emergency procedures. As would be expected, there are fewer 
interruptions in the actual launch (Figure 4) than in the intentionally difficult simulation. 

But the procedures are more out of order in the real launch than in the simulation. This 
means that the NTD must spend substantial time turning pages in the OMI searching for the 
step to be signed off. 

For a more detailed description of the behavior in the launch tapes, we define an utterance 
to be verbally spoken words, strings of words, or sounds (e.g., "uhmmm") bounded by 
100 ms or more of silence. We define a message , as used above, to be a sequences of 
utterances by a single speaker to a specific listener or listeners, without interruption by 
another speaker. We define a conversation as a sequence of messages between two or 
more specific speakers. We then transcribed each utterance, marked it with its speaker, 
and assigned it to a message and to a communication episode. (Appendix 1 contains about 
15 minutes of communications transcribed and marked in this manner.) 

The annotated transcriptions allow us to quantitatively describe some aspects of the NTD's 
job. One important aspect is the communications load The timed transcriptions permit a 
description of message traffic density on channel 212, as well as an idea of how much time 
the NTD must devote to communications. The unit of analysis is the message, or sequence 
of utterances that comprise one speaker's turn. The total elapsed time for the first launch 
attempt (25Feb90), measured from T-20 min to the abort at T-31 sec, was 30.2 min. The 
combined duration of all messages was 11.33 min. Thus, approximately 38% of the time 
there was message traffic on channel 212. For the 26Feb90 attempt, the tape was analyzed 
from T-20 min to the beginning of a lengthy, and final hold at T-9 min. The total elapsed 
time during this period was 14.82 min. Total message duration was 5.08 min, or 34% of 
the total time. A 23.89 min segment of the 28Feb90 launch contained 6.29 min of message 
traffic, yielding an estimate of 26%. Because of weather problems, this segment, and other 
segments from that launch containing long peri ods of s ilence as the launch team holds 
awaiting weather clearance, underestimates the normal percentage of message traffic. These 
analyses suggest that approximately one third of the time there is message traffic on channel 
212. 

The NTD is the direct participant in only some of these messages. For instance, for the 
25Feb90 launch attempt there were 271 messages of which the NTD participated in 166, or 
61%. For the 14 min segment of the 26Feb90 attempt there are 1 19 messages of which the 
NTD participated directly in 69, or 58%. For the 24 min segment of the 28Feb90 launch 
the NTD was involved in 105 of the 160 messages, or about 65%. Overall, approximately 
one third of the messages originate from the NTD. The NTD is clearly the most active 
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participant on the communication channel, and a substantial fraction of his time is devoted 
to communications. 

The NTD may be assumed to be passively monitoring conversations in which he is not 
directly involved (see Section 4), but a lower bound on the estimate of tim e spent on 
communications can be obtained by computing the time spent by an NTD in communication 
in which he is directly involved as either the initiator or recipient Of the 30.2 min elapsed 
time of the 25Feb90 launch attempt segment the NTD spent 5.68 min in direct 
communication, or approximately 19%. For the 14.82 min of the 2 6Feb 90 segment, the 
NTD spent 3.25 min in direct communication, or about 22%. The NTD was occupied with 
communications for 5 min of the 23.89 min of the 28Feb90 segment or about 20%. 

The mean time between messages can give an indication of how much contiguous free time 
is available to perform tasks other than communication. The mean time separating the start 
of one message from the start of the subsequent message for all members of the launch 
team in the three launch segments were: 6.7 secs for 25Feb90, 7.5 secs for 26Feb90, and 
8.4 secs for 28Feb90. Thus, message rate is roughly 8 messages per minute. For the NTD 
in particular, message rate is necessarily less. The mean time between message starts in 
which the NTD is either the sender or receiver is: 10.3 secs for 25Feb90, 12.8 secs for 
26Feb90, and 14.0 secs for 28Feb90. The NTD's message rate is roughly 5 messages per 
minute. The average duration of messages for all team members for the three launch 
segments are: 2.5 secs for 25Feb90, 2.6 secs for 26Feb90, and 2.4 secs for 28Feb90. 
Thus, for the NTD, the time between the end of one message and the beginning of the next 
(inter-message interval) is: 7.8 secs for 25Feb90, 10.2 secs for 26Feb90, and 1 1.6 secs for 
28Feb90. On average then, the NTD has about 10 seconds in which to accomplish other 
tasks between fielding messages. The distribution of inter-message intervals is positively 
skewed as a results of a few very long intervals, and for most of the time the NTD has 
significantly less than 12 secs between messages. 

One of the significant activity for the NTD that is done in the inter-message interval is 
marking steps complete in the OMI. Behavior graphs (Figures 3 & 4) have already shown 
that steps are not completed in the sequence in which they are presented in the OMI, 
making it necessary for the NTD to flip pages in the OMI in order to find the relevant 
procedure. We wanted to get some idea of how extensive this page turning was. 
Unfortunately, the videotape could not be used for this, since the order of step completions 
is constrained differently for T-20 min than for earlier portions of countdown, and because 
page flipping was often obscured in the videotapes. 

To estimate the amount of page turning in the OMI manual from the audio tapes, we 
calculated the distance in procedural steps (step size) from one step completion to the next 
in the transcription of the audio tapes. To get a rough idea of how this translates in to page 
turning proportions we computed the proportion of times a step size falls into a bin whose 
size is the number of steps visible on both pages of an opened manual. It turns out that this 
bin size is about 6; there are between 6 and 7 steps visible at any one time. Figure 5 plots 
the approximate proportion of times the NTD had to turn a given number of pages in order 
to find and mark a step completion that had just been given him. In the 30 minutes of 
elapsed time in which 176 steps were completed, the NTD had to traverse about 200 pages; 
eight times he had to flip 10 or more pages to find the step. The NTD's tell us that 
countdown segment beginning at T-20 min is a relatively well-behaved portion of the 
launch; other countdown stages have many more large transitions between steps. Indeed, 
the NTDs all complain about the excessive manipulation of the OMI, and are hoping that 
the improved hardware and software will relieve them of this particular bookkeeping task. 


12 


o 

c 


£ 8 . 

0) ^ 
Q. d) 
0) O) 

° O 

J2 - 

•S ® 

a* 

«•— 5 

2 = 
o « 
c ■*= 
o *"’ 
a 
o 



0123456789 10 + 

Number of pages flipped 


Figure 5. Estimated number of page flips required to locate the current step. 


The preceding analyses provide information about the activities of the NTD, how much 
communication, how much flipping of pages. Viewed in isolation, these activities appear as 
routine responses to local information. In actuality, each local behavior fulfills part of a 
larger unit of work. Just as sequences of utterances comprise a message, sequences of 
messages relate conceptually to larger scale operations. We have begun to identify higher 
level tasks in two ways. First, we reviewed the transcripts with an expert NTD, who 
connected together messages that he considered part of the same conceptual task. We then 
took this annotated transcript and compared it to section headings in the OMI that group 
steps by function, e.g., FUEL CELL PURGE contains steps 16-0908 and 16-0909. For 
each utterance, the NTD classified it as belonging to the same larger task uni t as the 
previous utterance, or belonging to a different larger task unit; for each utterance, we made 
the same judgment using the OMI section headings. In the segment of transcription 
partitioned by the NTD, there were 91 utterances, and thus 90 opportunities for the NTD's 
judgment to match the OMI headings. Of these 90 opportunities, the OMI headings agreed 
with the expert's opinion 77 times (85% agreement). 
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With this completed we now had a start on understanding what activities the NTDs 
perform, how much of their time is filled with these activities, and how the NTDs mentally 
represent their task. 

4. THE COMMUNICATIONS OF THE LAUNCH TEAM: 

A PRELIMINARY DISCOURSE ANALYSIS 

In the previous section, we examined the activities of the NTD as an individual in the 
launch process: the listening, speaking, page-turning, note-taking, etc. involved in doing 
the NTD's job. A large portion of the NTD's activities are communications with the other 
members of the launch team. In order to gain insight into the nature of these 
communications we performed a preliminary discourse analysis of a 15-minute segment of 
OIS communications (Appendix 1). 

Throughout this analysis, we define an utterance to be verbally spoken words, strings of 
words, or sounds (e.g., "uhmmm") by a single speaker to one or more particular listeners, 
bounded by 100 ms or more of silence. We define a message to be a sequence of one or 
more utterances by a single speaker to a specific listener or listeners, without interruption 
by another speaker. 

All communication on the OIS after T-20 minutes occurs within a general discourse 
situation (Figure 6a), where every member of the launch team monitors the same 
communication channel, Channel 212. Within the general discourse situation of universal 
passive monitoring, there are embedded discourse situations actively involving a subset of 
launch team members. Discourse situations may be interrupted, maintained as inactive, and 
returned to, but in practice, the resulting embedding seems to be rarely more than two deep. 

We have identified five types of discourse situations in which some launch team members 
play active roles. In an announcement (Figure 6b), one speaker makes an announcement, 
comprised of a single message, and all other members are active listeners. In a poll (Figure 
6c), one speaker conveys a request (a message) for information from a subset of the launch 
team. The members of that subset respond in turn with their own messages and are active 
listeners to both the request and the responses of the other members of the subset. In a 
two-party conversation (Figure 6d), a speaker and an active listener alternate roles and 
communicate messages (in a conversation, a message is equivalent to speaker s turn). In a 
two-party conversation with explicit overhearers (Figure 6e), several active listeners 
overhear a two-party conversation with the explicit knowledge of the conversation's 
participants. Such a conversation is explicitly set up with a management conversation 
(Figure 6f), with the NTD acts as facilitator, relaying a message to appropriate members of 
the launch team, who then ready themselves for the conversation. 

Specific announcements, polls, conversations with or without explicit overhearers, and 
management conversations, are referred to as communication episodes (or simply, 
episodes). Communication episodes are defined by the change from one discourse 
situation to another, where the discourse situation is defined by both its type, as described 
above, and by its participants. For example, a sequence of utterances where the NTD 
finishes a conversation with one person and then, without interruption, begins a 
conversation with another person, comprises two conversation episodes because of the two 
different sets of participants. 

The communications studied included approximately the first fifteen minutes after the 
countdown started at T-20 minutes on the first launch attempt This segment of 
communications contained 164 utterances, which combined to form 125 messages, making 


a. General discourse situation, passive monitoring of the OIS. 




b. Discourse situation for an announcement. One speaker (•) speaks to all 
others on the net. All others are active listeners (•). 



c. Discourse situation for a poll. One speaker requests information from a subset 
of the team, they respond in turn and are active listeners to the other responses. 




d. Discourse situation for a two-party conversation. 
Speakers alternate turns. 




e. Discourse situation for a two-party conversation 
with explicit overhearers. Speakers alternate turns 
and overhearers (©) actively listen. 


f. Discourse situation for a management conversation (below). For example, a request for a 
conversation with explicit overhearers comes to manager (M) from a speaker (SI ). M hails $2 
telling him to expect the conversation. M hails 01 and 02 telling them to be explicit overhearers. 
M becomes an explicit overhearer himself, 03, and Si and S2 converse. 






Figure 6. Discourse situations in OI S communication from T-20 minutes to launch. Each circle 
represents one member of the launch team. Although there arc hundreds of members of 
the launch team in totality, we have chosen to represent only a few for this illustration. 


H 
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up 42 communication episodes. These communication episodes included 6 
announcements, 0 polls, 32 two-party conversations, 3 conversations with explicit 
overhearers, and 1 management conversation. 

A conversation (two-party, with or without overhearers, or management) can be further 
described by the information exchanges 1 contained in it We have identified seven types of 
information exchanges evident in the segment of communications studied. The nominal 
form of a conversation begins with a Summons/Answer information exchange. In the 
summons half of this exchange, one speaker (SI) states the call-sign of the party with 
whom he or she would like to converse, then states his or her own call sign, and 
(optionally) the communication channel on which the conversation should take place. In 
the answer half of this information exchange, the party summoned (S2) answers by 
identifying himself or herself, optionally saying the words "Go" or "Go ahead" or 
identifying SI. After the S/A information exchange, the conversation continues with one 
or more of the six other types of information exchanges. A conversation of this type has 
no explicit end', rather, it is bounded by an extended pause or when someone else seizes the 
channel. 

The other six types of information exchange are as follows. 

Request/Promise/Registration -SI makes a request for information, S2 
promises to provide that information at a later time, and SI registers 
receiving the promise. 

Request/Compliance/Registration - S 1 makes a request for information, S2 
immediately complies with that request by providing that information, and 
SI registers receiving the information. 

Direction/Registration - SI directs S2 to do something and S2 registers 
receiving that direction. 

Assertion/Registration - S 1 makes an assertion and S2 registers receiving that 
assertion 

Offer/Acceptance/Compliance/Registration - SI makes an offer to provide 
information, S2 accepts the offer, SI provides the information, and S2 
registers receiving that information . 2 

Offer/Refusal/Registration - SI makes an offer to provide information, S2 
refuses the offer, SI registers receiving that refusal. 

These seven types of information exchanges are sufficient to nominally classify over 90% 
of the information exchanges in the segment of communications studied. Of 85 
information exchanges in the 36 conversations studied, there were 39 Summons/Answer, 

0 Request/Promise/Registration, 8 Request/Compliance/Registration, 13 Direction/ 

Registration, 18 Assertion/Registration, 0 Offer/Acceptance/Compliance/Registration, 

1 Offer/Refusal/Registration, and 6 unclassifiable information exchanges. Of the 6 
unclassifiable information exchanges, 3 were too incomplete to allow classification, i.e., 

requests that received no response so they could not be classified as either 


1 Information exchanges are derived from the adjacency pairs described in Clark, 1985. In fact, the 
Summons/ Answer information exchange is identical to the Summons/Answer adjacency pair. The other 
information exchanges can be broken down into strings of adjacency pairs. For example, the 
Request/Promise/Registration information exchange is a Request/Promise adjacency pair where the message 

that is the promise doubles as the end of that adjacency pair and the beginning of a Promise/Registration 
adjacency pair. If an Assertion/Registration pair is added to the eight described by Clark, the other five 
information exchanges arc also combinations of adjacency pairs. We have chosen to work at the grain size 
of information exchanges because it is less cumbersome than strings of adjacency pairs and adequate to 
describe the conversations found in these data. 


Communication Episode 
Number Type 


Utterance 

Number 


1 1 


Announcement 


Announcement 


2-way conversation 



Announcement 


2-way conversation 



2- way conversation 



2-way conversation 



2-way conversation 



2-way conversation 



2-way conversation 


2-way conversation 


10 
10 
1 1 
12 

13 

14 

15 

16 
17 

17 

18 
19 

19 

20 
20 
21 
22 

23 

24 


Comment 


1 


1 -6 Assertion — nominal announcement 
7 Assertion nominal announcement 
Summons — i 

nominally SAORR 
(missing Answer 
inserted Promise) 


Offer 
Refusal 
Promise 
Registration 
Assertion — 

Summons — 

Answer 
Assertion 
Registration* 1 
Summons — i 
Request nominal SARCR 
Compliance - ^ (missing Answer) 


nominal announcement 
nominal SAAR complete 


1 



Summons 
Direction 
Answer 
Registration 
Summons 
Answer 
Direction 
Summons' 
25,26 Answer 

27 Assertion 

28 Registration* 1 

29 Summons — 

29 Direction 

30 Answer 

30 Registration 

3 1 Summons 
31 Request ” 


nominal SADR complete 


nominal SADR 
(missing Registration) 

nominal SAAR complete 


nominal SADR complete 


nominal SARCR? 

SARPR? •<; 
(missing other party) 


Figure 7. Examples of communication episodes for a portion of the OIS communications 
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Request/Compliance/Registration or Request/Promise/Registration. The other 3 were not 
strictly classifiable under this scheme, but fit into the spirit of the analysis, one was 
nominally an Offer/Refusal/Registration but the refusing party promised to make a request 
at a later time, and 2 others were nominally Requesl/Compliance/Registration but the 
complying party first registered the request and then immediately complied. Figure 7 
shows a diagr am of some of these communications with the breakdown mto episodes 
(announcements and conversations) and information exchanges. 

The component messages of information exchanges (Summons, Answer, Request, etc.) 
are called speech acts. For each of these speech acts, we have determined the nominal form 
of the speech act, the preconditions for that speech act to occur in the OIS communication, 
and the postconditions existing in the discourse situation after the speech act has occurred 
(Appendix 2). The form includes typical types of illocutionary acts (e.g. commissives, 
which commit the speaker to perform some future actions, or assertives, which express 
beliefs) that make up these messages, specific commonly used words or phrases, and 
necessary or optional content The preconditions are what the speaker must believe about 
the state of the communication channel and the launch situation before uttering that speech 
act is warranted; there is no guarantee that these conditions actually exist in the world, 
simply that the speaker believes they do. The postconditions do exist in the world prior to 
the occurrence of the speech act. The form, preconditions and postconditions for OIS 
communication distinguish the use of these speech acts from the use of similar speech acts 
in other discourse situations with other characteristics (e.g., different bandwidth like face- 
to-face communication, different roles like teacher/student or doctor/patient). 

We combined the structure evident from the analysis of the discourse situations, 
conversations, information exchanges, and speech acts, with the information in the OMI 
and our knowledge of the NTD's task, to infer a goal hierarchy for the OIS 
communications. At the top of the hierarchy is the goal to launch the space shuttle. This 
goal can only be fulfilled when all of the sections in all the sequences of the OMI are 
accomplished. Our study focuses on the particular sequence used after T-20 minutes, 
Sequence 16, which is made up of 317 sections (15 of which are for contingencies only, 
not executed under normal conditions). These 317 sections cover 1132 steps, which are 
the smallest units of work to be handled by the launch team. Many steps can be completed 
with a single communication episode, but some require a series of conversations about the 
same topic. Considering the simplest case (one communication episode completes one 
step), in any one communication episode, the goal of the first speaker is usually to 
complete a step. If that goal is met, the step is complete; if it is not, the step is said to be 
advanced. For example, consider the conversation and excerpt from the OMI in Figure 8. 


Speaker Message Speech Act 

NTD: SRO NTD, 212 Summons 

SRO: NTD SRO go ahead sir Answer 

NTD: Verify if 70 minute jimsphere has normal track Request 

SRO: Yes sir, it has normal track Compliance 

NTD: Copy Registration 


SEQ_ TIME 


CMP RESP DESCRIPTION VER 1E. 


16-0960 -15M00S 

I-25M00S 


NTD SRO VERIFY 70 MIN JIMSPHERE HAS NORMAL 

212 TRACK 


Figure 8. Conversation and OMI excerpt for step 16-0960. 


Goals; 

Launch Handle section 


Handle step 
i(number) 

Advance step 

Utterance Number 



Assertion 

Assertion 

Summons 

Offer 

Refusal 

Promise 

Registration 

Assertion 

Summons 

Answer 

Assertion 

Registration 

Summons 

Request 

Compliance 

Summons 

Direction 

Answer 

Registration 

Summons 

Answer 

Direction 

Summons 


25,26 Answer 

♦27 

Assertion 

♦28 

Registration 

♦ 29 

Summons 

♦29 

Direction 

♦30 

Answer 

♦30 

Registration 

♦31 

Summons 

♦31 

Request 


Figure 9. Goal structure of a portion of the OIS communications 
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The goal of the NTD is to obtain compliance with his request for information, and, in this 
case, SRO complies immediately with the necessary information; the registration ("copy") 
completes the step. Alternatively, if the SRO did not have this information available, he 
would have promised to provide it when it became available some time in the future. This 
would not have completed the step, but advanced it. In our preliminary study the only 
communication episode that advances a step is a conversation in which a Promise speech 
act appears. On the other hand, a step can be completed with any of the types of 
communication episodes: announcements, polls, conversations, etc. Conversations that 
complete a step contain at least one of the information exchange types other than 
Summons/Answer, which is common to all conversations, and 
Request/Promise/Registration. 

As our earlier problem behavior graphs showed, steps can be handled out of sequence, and 
several sections are often begun but not completed at any one time. In addition, several 
steps could be advanced or completed within a single conversation, but the completion of 
any particular step occurs in a single conversation. Figure 9 displays the goal structure as it 
is traversed in OIS communication. This communication segment begins with two 
assertions that complete steps 16-0906 and 16-0907, which complete the section RESUME 
COUNT. The conversation completing step 16-0910 also contains a promise that advances 
step 16-1043 and opens the goal of completing the section that contains that step. In all, 
this small segment of communication, approximately 2 minutes, initiates and completes 
goals to handle 1 1 steps, advances 1 step, completes 5 goals to handles sections and leaves 
2 section goals active for completion in the future. 

As presented, this goal hierarchy contains the goals of the entire launch team, and describes 
communications on the OIS for all members of that team. However, any one individual's 
goal hierarchy is probably similar to the team's hierarchy. For instance, this goal hierarchy 
and the explicit description of the pre-and post-conditions for each of the speech acts 
applied to a particular conversation clarifies some of the deviations from the nominal form 
of a conversation. In one communication episode, the NTD says, "Houston Flight NTD, 
perform BFS preflight uplink loading", and FLT replies, "In work". This is nominally 
encoded as Summons/Answer then Direction/Registration, but the answer to the summons 
is missing from the conversation. The missing answer can be explained as follows. The 
goal of the NTD is to complete step 16-0933. Completion of this step requires that 
Houston Flight receive the direction to PERFORM BFS PREFLIGHT UPLINK (as written in the 
OMI) and commit to doing it, and that the NTD knows that Houston Flight received and 
commited to performing this step. Assuming that all the pre-conditions are met, the chain 
of post-conditions for the nominal structure of a conversation that completes this step result 
in just those conditions in the world (Figure 10a, refer to Appendix 2 for complete details). 
With incomplete information exchanges, the preconditions help establish the desired state. 
A precondition of a registration of a direction is that the registering party be authorized to 
carry out that direction. The OMI shows that only Houston Flight is authorized to carry out 
the direction; the fact that a registration was provided at all implies that the pre-condition 
was met, and that FLT is indeed the registering party. Thus, the post-conditions imply that 
the desired state was achieved (Figure 10b). 

Since an answer is not required for completion of the goal, including an answer in the 
conversation may seem inefficient and the question as to why it is used at all (as opposed to 
why it is NOT used, occasionally) may be raised. Our surmise is that the inference process 
required when an answer is not used is perceived to be less reliable than the simple 
recognition of post-conditions necessary in an complete exchange. The necessity of high 
reliability in the launch procedures has influenced the communication culture, producing a 
preference for highly reliable conventions for conversation. In this preliminary analysis. 
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Summons — > 

^ POST: 

Answer 

/ 

^ POST: 

Direction 

"VOST: 

Registration . 

J 

POST: 


Figure 10a. Logic chain through a complete information exchange 

Summons 


Direction 


Registration 


POST : NTD active participant 

POST: FLT has opportunity to hear direction 

PRE: Registering party is authorized 

According to OMI, only FLT is authorized to do this direction 

Infer, therefore, that registering party is FLT 


POST: FLT committed to fulfilling direction 

Figure 10b. Logic chain through an incomplete information exchange 

Figure 10. Logic chains through speech acts that accomplish a goal with complete and 
incomplete information exchanges. 


27% (23) of the 85 information exchanges are incomplete (in 13 of the 36 conversations 
studied). Of the 13 conversations, 4 conversations (representing 4 incomplete information 
exchanges) are missing only an answer to a summons like the example above and complete 
the goal of the conversation in spite of the incomplete. Five conversations (representing 12 
incomplete information exchanges) are missing an entire side of the conversation. Another 
4 conversations (representing 7 incomplete information exchanges) are missing the final 
registration in the conversation, so thaf the completion of the goal is not assured. These 
incompletions are yet to be explained. 

The goal structure can also be used to understand some of the fine detail of the OIS 
communications. For instance, in the segment of OIS communication we studied, there 
were 73 referents to be resolved. The resolution of all of these referents present no 
problem to the participants because there are no exchanges to clarify referents (e.g. there 
were no exchanges like SI: "Copy that", S2: "What was that you copied?"). A possible 
explanation for the ubiquitous ease of referent resolution may be that referents usually refer 
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to things within a single goal. Of the 73, over 80% (60) are resolved within the confines of 
the current conversation in service of a single goal. Only 2 are resolved by reference across 
a conversational boundary, and both of these are within the completion of a single goal (a 
step requiring multiple conversations). An additional 3 referents are resolved from visual 
information in the OMI related to the step that is the current goal. Of the 8 resolutions 
requiring knowledge outside the current step-handling goal, 4 occur in conversations about 
the amplitude of the shuttle pilot's communication channel and refer to the amplitude of an 
utterance, 1 is a reference to "today" which requires general world knowledge, 2 refer to 
the previous speaker in the same conversation although there is a change of step- handling 
goal, and the remaining 1 requires an understanding of the mode of communication (i.e., in 
"I did not copy your last", "last" refers to the last verbal message uttered by the partner in 
the conversation). Since the resolving information is almost always available within the 
same goal, the goal structure may be an aid in resolving referents. 

The analysis of these data in light of the task of launching a space shuttle, have produced 
categories of discourse situations, conversations, information exchanges, and speech acts 
and an inferred goal hierarchy that connects them. From these emerge an instantiation of 
Grice's maxims of cooperative communication applied to the particular domain of OIS 
communication shortly before a space shuttle launch. This instantiation can be expressed 
as follows (adapted from the description of Grice's maxims in Clark, 1984). (Throughout 
the OIS maxims below, the terms in all upper case refer to the contents of those columns in 
the OMI: CMD refers to the person who initiates the communication, RESP refers to the 
person who receives the communication, SEQ refers to the step number, DESCRIPTION 
refers to the description of the step, etc.) 


Grice's Maxims 

Cooperative principle 

Make your conversational contribution such 
as is required, at the stage at which it 
occurs, by the accepted purpose or 
direction of the talk exchange in which you 
are engaged. 

1. The maxim of quantity 

a. Make your contribution as informative 
as is required (for the current purposes of 
the exchange) 


OIS Maxims 

OIS Cooperative principle 

Make your contribution to the launch 
preparation, as soon as is safely possible, as 
written in the OMI. 


1. The maxim of quantity for the OIS 

a. Make your contribution as informative as 
is required 

i. assume the person to whom you are 
speaking can reference the correct step in the 
OMI either by its SEQ or by its 
DESCRIPTION, so communicate either the 
SEQ or DESCRIPTION in each exchange. 

ii. assume the person to whom you are 
speaking cannot reference the correct step in 
the OMI with just the CMD, RESP, or 
TIME, so do not use any of these as a sole 
reference point 

iii. since OIS communication is audio- 
only, the listener must inform the speaker 
verbally that his or her message was received 
and understood. Therefore, answer each 
summons and register each message. 
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b. Do not make your contribution more 
informative than is required. 


2. The maxim of quality: 

Try to make your contribution one that is 
true. 

a. Do not say what you believe to be false. 

b. Do not that for which you lack adequate 
evidence. 


3. The maxim of relation: 
Be relevant. 


4. The maxim of manner: 

Be perspicuous. 

a. Avoid obscurity of expression. 


b. Avoid ambiguity. 


c. Be brief (avoid unnecessary prolixity). 


b. Do not make your contribution more 
informative than is required 

i. do not communicate both the SEQ and 
DESCRIPTION. 

ii. do not communicate the TIME unless 
explicitly specified in the DESCRIPTION of 
the step. 

iii. all communication is over Channel 212 
at this time in the countdown, and everyone 
knows this, so do not communicate the 
CHANNEL. 

iv. since the communication is only verbal 
over the OIS, t he CMP shoul d communicate 
both who he or she is and to whom the 
information is addressed (the RESP), as 
written in the OMI. 

2. The maxiin of quality for the OIS: 
Try to make yo ur co ntribution one that is 
true, (unchanged from Grice's maxims) 

a. Do not say what you believe to be false. 

b. Do not that for which you lack adequate 
evidence. 


3. The maxim of relation for the OIS: 
Be relevant 

i. information from a CMD is only 
explicitly relevant to the RESP, as written in 
the OMI. 

ii. Since the communication is only verbal 
across the OIS, a CMD should summon the 
RESP and receive an answer to ensure that 
the relevant person is actively listening, 
before passing the information. 


4. The maxim of manner for the OIS: 
Be perspicuous. 

a. Avoid obscurity of expression. 

i. when saying the DESCRIPTION of a 
step, use the words written in the OMI, or a 
very close approximation. 

b. Avoid ambiguity. 

i. do not say the DESCRIPTION of one 
step and the SEQ of another step in the same 
turn, because the RESP may think you are 
confusing the two as one. 

c. Be brief (avoid unnecessary prolixity), 
i. when saying the DESCRIPTION of a 

step, use only the words in the OMI, or a 
very close approximation, do not embellish. 
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d. Be orderly. d. Be orderly. 

i. communicate the completion of a step as 
soon as it is complete, or, if several steps are 
related under a single section in the OMI and 
are completed within a short time of each 
other, wait until the group is complete and 
then communicate the completion of all the 
related steps. 

ii. when giving the completion of several 
steps, give them in chronological order. 


As with Grice's maxims for general conversation, the OIS maxims are not inviolate. For 
instance, the N I L) routinely includes the channel number when he makes a summons 
although it is more informative than is necessary for the T-20-min-to-launch OIS 
communication situation (because all communication is on Channel 212). Prior toT-20 
min, communications happened on many channels, and indeed, it is part of the NTD's job 
to direct conversations to different channels. Thus, changing communication situations 
dictate changing maxims. Communication styles from one situation may be earned into 
another situation without disruption of the cooperative principle, although they may violate 
small points in the relevant maxims. 

The OIS Cooperative Principle and related maxims of conversation give prescriptions about 
what content to communicate, when to communicate it, and what words to use to convey 
that content. These, together with the inferred goal structure, the speech acts, information 
exchanges for this portion of the countdown, the form of conversations, and discourse 
situations, provide much information that could be used in constructing a computational 
model of the NTD and evaluating the design of computer support aids. This information 
presented here is tentative, in that it is derived from a preliminary discourse analysis of only 
15 minutes of communications. Before using this information extensively in modeling or 
design evaluation, further work must be done to confirm the validity of these results: 
analysis of a greater portion of communication, investigation of non-routine OIS 
communications to determine if and how the OIS maxims are violated or changed (e.g., 
during launch simulations where many problems are encountered), differences in the 
maxims with for different launch situations (e.g., prior to T-20 min and after T-20 min) or 
launch team members, experiments to validate the maxims with real NTDs, and comparison 
to other discourse situations to more fully understand the unique character of the OIS 
communication task. However, a preliminary version of a computational model, with 
rudimentary incorporation of some of the results of this discourse analysis, is proposed for 
discussion in the next section. 

5. THE COMPUTATIONAL MODEL: NTD-SOAR 

We are using our understanding of the NTD's task to build a computational mode l of the of 
the NTD. That is, we are designing a computer program that will mimic the NTD's 
behavior in many situations, reproducing the functions the NTD performs, in the same 
amount of (simulated) time, within human p ercep tual and information processing 
capabilities. Our model currently views the NTD as an individual information processing 
agent This agent gets inputs from the environment, primarily the presentation of 
information through verbal communication and visually from the OMI, and produces 
outputs that effect that environment, primarily saying information over the communication 
channels and writing notes in the OMI. The agent's outputs are produced in the service of 
goals, through operators that manipulate symbolic representations of elements in the 
environment The choice of operators and the sequence in which they are used are 
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constrained by the agent's knowledge, including knowledge of the environment, of desired 
states for the environment, of mechanisms for change, and of communication (OIS 
maxims). The current version of the model performs the routine cognitive skills involved 
in performing non-problematic communications on the OIS, finding the appropriate 
location in the OMI, and checking off steps in the OMI as they are complete. Such a 
computational model of routine cognitive skill, where performance is driven by goals and 
implemented with operators constrained by knowledge, is one in the family of GOMS 
models often used in HCI research (Card, Moran & Newell, 1983; Olson & Olson, 1990). 

Our first few versions of this model will assume that the information coming into the NTD 
comes in perfectly, that is, the model's perceptual system will make no errors in perceiving 
information. Likewise, the first few versions of the model will assume that the information 
flowing from the model will be executed perfectly, that is, the model will not mix up words 
in its speech, hit an unintended button, or cross out the wrong step in the OMI. Eventually, 
we expect to model perceptual errors and action slips, but it is important to get the model 
working correctly with perfect information before attempting to assess the effects of 
imperfect information. 

The model also does not take into account the source of information, that is, it does not 
attach any meaning to a piece of information attributable to its source, rather than its 
content. Thus, all sources are treated as equally accurate, equally trustworthy, equally 
timely, etc. This is true of human sources of information as well as inanimate sources of 
information. For the most part, this is a reasonable assumption for the initial computational 
model because the parties authorized to complete each step are recorded in the OMI and, in 
the vast majority of cases, the OMI is followed to the letter in this regard. In the hour of 
communications studied, we have only seen one instance of a message uttered by a person 
unauthorized to do so. In that case (utterance 6 in Appendix 1), CGLS says "T-20 minutes 
and counting" which usually completes step 16-0907. However, only NTT) is authorized 
to complete that step, and he does so immediately, saying "Step 907 verified complete". 
Since this type of exchange happens so rarely, we have chosen to ignore it for the purposes 
of a baseline computational model. 

Our GOMS model of the NTD is built within the Soar unified theory of cognition. 2 We will 
first present a brief overview of the Soar unified theory of cognition, and then a description 
our preliminary model, NTD-Soar. 

5.1 A Brief Overview of the Soar Unified Theory of Cognition: 

Soar is a recent attempt to provide an architecture for human cognition (Laird, Newell & 
Rosenbloom, 1987; Newell, 1990). Soar is given in the fashion of a programmed 
computer, with data structures, memory accessing organization, and full details of the 
operation of the processors. Thus, one can specify the contents of Soar memory structures 
for particular users in particular task situations. TTiese contents, in effect, program Soar so 
that it produces simulations of the behavior of the user in the task. 


As succinctly described in (Lewis, et al., 1990) and in more detail elsewhere (Laird, et. al., 
1987; Newell, 1990), the Soar architecture formulates all tasks in problem spaces, in which 
operators are selectively applied to the current state to attain desired states. Problem solving 
proceeds in a sequence of decision cycles that select problem spaces, states and operators, 
resulting in the application of the operator to move to a new state in the space. Each 
decision cycle accumulates knowledge from a long-term recognition memory (realized as a 


2 See John, Vera, & Newell, 1990 for a discussion of the relationship between GOMS and Soar. 
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production system). This memory continually matches against working memory, 
elaborating the current state and retrieving preferences that encode knowledge about the 
next step to take. Access of recognition memory is involuntary, parallel, and rapid (on the 
order of 10 msec). The decision cycle accesses recognition memory repeatedly until 
quiescence, when no more knowledge can be brought to bean then the decision is made 
about which step to take next Each decision cycle takes on the order of 100 msec. 3 

If, at quiescence, the accumulated coded knowledge in working memory is insufficient (or 
conflicting), so that Soar's next step cannot be determined, then an impasse occurs. Soar 
responds to an impasse by creating a subgoal in which a new problem space can be used to 
acquire the needed knowledge (or resolve the conflict). If, similarly, lack of knowledge 
prevents progress in this new space, another impasse occurs and another subgoal is created 
-- and so on, leading to an entire goal-subgoal hierarchy. Once an impasse is resolved by 
problem solving, the chunking mechanism adds new productions to the recognition 
memory that encode the results of the problem solving. Thus, the impasse is avoided in the 
future, because these productions provide the appropriate knowledge immediately. 

Soar interacts with the external environment through perceptual and motor processes, 
which operate through the working memory. Incoming perceptions are added to the current 
state in the top problem space and motor commands are made part of this current state. The 
interactions occur asynchronously with the operation of the cognitive decision cycle. 

Soar is sufficiently complete and plausible for human cognition to be used to model and 
explain many diverse cognitive phenomena (Lewis, et al. 1990; Newell, 1990). Examples 
include natural language comprehension, problem solving, immediate reasoning, perceptual 
search, strategy discovery and change, and the taking of instructions. These tasks 
encompass the major dependent variables of interest in HCI -- time, errors, and learning 
rates. 

5.2. The Structure of NTD-Soar 

From the observation, interviews, descriptive and discourse analyses of the NTD s 
behavior de tail ed above, we posit that the NTD's job requires three types of activities. 1) 
routinely following a procedure in the QMI, 2) handling an anomaly from standard 
procedure, and 3) updating an internal model of the the state of the space shuttle, its 
associated systems (e.g., launch pad), and the countdown. Each of these will be discussed 
briefly to clarify what we mean by these terms and from what data we inferred them. 

The bulk of the observed behavior is explicitly involved in following procedures in the 
OMI. When the NTD is summoned, he answers and enters into an information exchange as 
described in Section 4. That information exchange directs him to the relevant step in the 
OMI, where he takes appropriate action (e.g., checking of a completed step, filling in 
required information). When that conversation complete, the NTD either receives another 
summons, or he looks ahead in the OMI and initiates communications to complete the next 
step in which he is involved. This behavior seems to be a routine cognitive skill, a local 
response to incoming auditory information or information written in the OMI. The 
information exchanges in this routine part of his task have well-known nominal formats, 
and are not difficult for the NTD to perceive or understand. Unlike some anomalous 
situations that may arise during countdown, when the launch is progressing smoothly the 


3 The recognition memory access time and decision cycle time given are estimates of how long these 
processes would take in the human information processor, not the real time for a computer simulation of 
those processes to run (current computer technology is much slower). 
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NTD does not seem to be involved in any problem solving, difficult comprehension of 
ambiguous information, or interpretation of goals; he is simply following a procedure at 
which he has become expert 

For example, consider the following short exchange between the NTD and Flight (FLT) at 
Johnson Space Center (responsibility for the shuttie transfers from KSC to FLT 
immediately after a launch, but the two centers are in constant communication during the 
countdown). The numbers indicate start and end times of the utterance in seconds. 


20.593 

21.533 

NTD: 

Houston Flight, 

21.666 

22.080 


two one two. 

22.080 

23.866 


we got a requested weather briefing 
at this time. 

25.267 

28.184 

FLT: 

We will not need that one. 
We'll have an update at T-9. 

28.448 

28.750 

NTD: 

Copy. 


This exchange occurs just after the countdown has resumed at T minus 20 minutes 
(following a ten-minute built-in hold). After confirming the resumption of the countdown 
over the public address system (step 16-0907, Figure 2), the NTD looks through the OMI 
for the next step in which he plays a part, step 16-0910 on the next page. The step is an 
optional weather briefing, to be requested by FLT. When the NTD does not receive a 
request from FLT, as he expects from the procedure written in the OMI, he checks with 
FLT as to whether the step is to be performed. FLT responds that it is not 

To model this exchange, NTD- Soar, perceives via simulated audio input that the count has 
resumed at T minus 20 minutes and matches the content of that announcement to the 
content in the OMI DESCRIPTION column to place itself at the right location in the OMI 
(OIS maxim Lari) NTD-Soar then searches the OMI to find out what he needs to do next. 
It finds that step 16-0910 is the next step in which the NTD is involved and reads the 
information associated with the step. It reads that the step is still pending, and that FLT is 
due to say whether it will be performed. NTD-Soar forms the goal to complete the step. 
NTD-Soar then hails FLT (via simulated speech output), according to OIS maxim 3, with a 
Summons of the form shown in Appendix 2. It then orients FLT to the step in question by 
saying a close approximation to the DESCRIPTION of that step in the OMI (OIS maxims 1 
and 4c), indirectly offering the weather information. On hearing FLTs refusal, NTD-Soar 
crosses off the step as not required (via simulated hand action), and acknowledges FLTs 
refusal by saying "Copy" (OIS maxim la). It finishes the task, by crossing off die 
remainder of the steps in the subsection since they are only necessary if the weather 
briefing has been requested. 

Occasionally, an anomaly occurs and the launch process deviates from the procedures in 
the OMI. Anomalies range from mild, well-understood problems, to crises that forte a 
return to a previous point in the countdown, a scrub of die launch, or emergency 
procedures. Many anomalies have been thoroughly analyzed ahead of time, so that 
procedures for handling them are printed in books similar to the OMI. In these cases, the 
handling of the anomaly is similar to following an OMI procedure; it involves finding the 
correct place in the correct book and following the procolure for that situation. When the 
anomaly lies outside the range of what has been previously thought through, the NTD 
seems to be gathering necessary information and then following well defined procedures, 
or if no procedures exist after all information is exhausted, calling for a scrub of the launch. 
We determined this part of the task analysis from interviews with the NTDs and with the 
training team member who performs the launch simulations for NTD training purposes. 
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In the course of following the procedures in the OMI, the NTD seems to acquire, and is 
" able to access, immense amounts of information about the state of the shuttle systems and 
the countdown process. Evidence for this is that the NTD does hot only respond to the 
incoming information or go to the next step in the OMI, but he also initiates steps out of 
older, calls for information about incomplete steps, and flips pages well ahead of the 
current information during slow times on the communication channels (observed with 
videotapes). We infer that these behaviors reflect at least two types of information: 
information about the physical state of the shuttle and associated systems (e.g., whether the 
launch pad is cleared of personnel, whether the liquid hydrogen fuel has been loaded, etc.), 
and information about the timecourse of the launch countdown (e.g., whether procedures 
that must be done within a certain time of each other are progressing apace, whether the 
current progress of the countdown will allow the launch to occur during an acceptable time 
determined by satellites passing overhead or approaching weather, etc.). There is too much 
information to be held in the NTD’s working memory, so we assume it is stored in long- 
term memory or external memory (e.g. the log book). The NTD probably constructs and 
maintains a "mental model" of the state of the shuttle, its subsystems and the countdown, 
that allows him to access and reconstruct the information necessary to perform his job. The 
information in this mental model affects how the OMI procedures are followed, and the 
information gotten by following OMI procedures effects what's in these models. The 
current version of NTD-Soar does not yet include the mechanisms for producing these 
model updates from the local OMI procedure following. However, several approaches to 
this learning-by-doing have been explored in other Soar systems (Lewis, 1991; Young, 
1991) and we expect to be able to modify these approaches for our task. 


6. DISCUSSION 

We have described of the NTD's job and some analyses of observed auditory 
communications. We have also presented a computational model of the portion of that job 
that involves routine, skilled performance. Here, we will demonstrate how such a model 
can be used to evaluate hypothetical computer support systems, both with regards to the 
NTD's individual performance and the entire launch team's performance. We will then 
examine the value added to the computational model by the discourse analysis and speculate 
as to how discourse analysis could be used in design. 

6.1. W/wr-z/Evaluations on the Individual Performance Level 

NTD-Soar currently performs the routine cognitive skill of following procedures written in 
the OMI. It uses simulatied auditory input and knowledge of the organization of the OMI 
to locate the correct place in the OMI and perform the appropriate action (crossing off a 
completed step, filling in a write-in section, etc.). We believe that this routine cognitive 
skill is but one aspect of the NTD's performance, with the more global issues of 
understanding how each completed step advances the shutde system and the countdown 
being even more important than these routine interactions. Therefore, we believe that the 
model must integrate the local performance issues with the global understanding before any 
complete evaluations of changes to the NTD's computer support systems can be made. 
However, the current model of local performance is sufficient to demonstrate one possible 
style of evaluation that provides information about the relative efficiency of support 
systems with respect to performance time. 

Consider the following two exchanges (Figure 11) between the NTD and the entire launch 
team over the public address system (PA) and, seven seconds later, the exchange between 
the NTD and Houston Right Control (FLT) about the weather briefing examined 
previously. 
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Utt.no. Start (s) 

End fs) 

Speaker 

Channel 

Utterance 

u7 

12.009 

13.347 

NTD: 

PA 

Step 907 verified complete 

u8 

20.593 

21.533 

NTD: 

212 

Houston Flight, 

u9 

21.666 

23.866 



two one two we got a requested 
weather briefing at this time. 


SEQ 

TIME 

CMD 

RESP 

DESCRIPTION 

YER1F, 

16-0910 

-20M00S 

‘FLT 

NTD 

WEATHER BRIEFING REQUESTED 

34-01 


COUNTING 

212 


BY *FLT. 




NTD 

STM 





212 

JYES 

YES NO 



Figure 1 1. Conversation and OMI exceipt for step 16-0910. 


Prior to these exchanges, the countdown was restarted at T-2Q minutes and u7 represents 
the NTT) completing his responsibility for step 16-0907 by stating that fact over the public 
address system and crossing off that step in the OMI. The n ext role for t he NTD is on the 
next page in the OMI (Figure 11). The NTD is fisted as the RESP, the person who 
receives the call, on a call from FLT in which FLT requests a weather briefing. However, 
instead of being the RESP as is written in the OMI, the NTD initiates the call to FLT and 
offers the weather briefing. We model this behavior in NTD-Soar with a Get-Next- 
Participation operator that scans the CMD and RESP columns in the OMI from the point of 
the last participation (in this case, where the NTD marked off step 16-0907 on the previous 
page) to the next pl ace w here die NTD is either CMD or RESP. When the next 
participation is found, NTD-Soar then looks at the associated time of the participation. If 
the time is the same as the present countdown clock, NTD-Soar initiates a summons to the 
other participant, irrespective of whether the NTD was the intended initiate or the intended 
receiver in that step. After the NTD’s summons is answered, if the NTD is the CMD, 
NTD-Soar gives the step content printed in the OMI. If the NTD is the intended RESP, as 
in this example, NTD-Soar gives the step-content printed in the OMI in the form of an 
offer, notifying the intended CMD that the NTD is prepared to complete the step. 

The scanning behavior leading up to the exchange with FLT currently involves scanning 
down columns in the OMI, looking for the initials "NTD", and turning pages. The 
activities associated with this behavior can be represented in a critical path method (CPM) 
schedule chart. Figure 12. The CPM chart shows the operations in the model that seek and 
find the next point of NTD participation. Each box represents a perceptual, cognitive or 
motor process (fined up horizontally), and each line represents an information dependency 
between the processes. These processes proceed sequentially or in parallel, as dictated by 
the information dependencies. For example, there is a dependency fine between the 
cognitive process that initiates an eye movement and the motor process that performs the 
actual eye movement, because the deliberate movement cannot occur before the intention to 
move has been formed. Each process is assigned a duration, in ms, appearing above the 
upper right comer of each box. These durations are estimated from previous research and 
the actual content of the process. From these ti me es t im ates and the information 
dependencies, the critical path (the total time for all the processes to be complete) can be 
calculated. In this case, all the processes proceed serially and the total time is 31 10 ms. 
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This model of the scanning behavior can now be used to evaluate a different interface to the 
information in the OMI. As an illustration, imagine an on-line OMI, where the sequences 
in the OMI are reproduced on the CRT screen with the same column al formatting as in the 
paper version. Imagine also, a function key that puts the next sequence in which the NTD 
participates at the top of the screen. Then, instead of scanning down columns and turning 
pages to find his next participation in the countdown, the NTD moves his finger 
horizontally until it is over the correct key and presses the key. After some system delay, 
the display changes, putting the NTD's next sequence at the top of the screen. The 
interaction with this interface, from the point where the intention to find the next 
participation sequence is formed until the point where the NTD recognizes that he has 
access to the information he desires, is represented in the CPM chart in Figure 13. With 
this interface, the eye-movement to the standard place on the computer screen can be done 
in parallel with waiting for the display to be presented, and the total time to find the next 
point of NTD participation is 890 ms plus the system response time of the computer 
support system. 

The two CPM charts can now be compared and several what-if evaluations can be made. 
For example, in this particular scenario, the next point of NTD participation is three steps 
and one page away from the previous point of participation. Given that scenario, the on- 
line support would need to have a system response time of less than 2220 ms 
(3110-890 ms) to be used as quickly as the current OMI. What if the average distance 
between steps requiring NTD participation were only 2 steps and included a page turn only 
half the time? Then the average OMI critical path would have one less eye-movement and 
perception and a page turn only half the time and the average total time would be 2080 ms. 
Therefore, the system response time for the computer tool would have to be less than 
1 190 ms to be as effective. What if further analysis revealed that the NTD used not only 
the specific information in the sequence in which he was participating, but he also used a 
sense of the distance between the current step and the next step, in understanding the status 
of the countdown. A computer-support application designer might then consider using 
animation to simulate the information passing in front of the NTD's eyes as he scans 
through the current OMI, to provide continuity of information and the necessary distance 
information. Research into animation in information display indicates that a transformation 
from one information state to another should take about a second; if it is much shorter, the 
user loses object constancy and has to reorient himself. If [it is] much longer, then the user 
gets bored waiting for the response" (Card, Robertson & Mackinlay, 1991). The CPM 
chart indicates that a system response time of about a second does not increase the 
performance time beyond the paper OMI, so animation would be a viable option. The 
interface designer would then need to determine whether the distance between steps should 
be represented as a function of response time, as it is with flipping pages and scanning 
them, or whether the density of information passing before the eyes in a constant amount of 
time would give the same useful information to the NTD. Further what-if analyses would 
help in this and other design choices. (An example of such comparisons in actual 
workstation evaluation for telephone operators is presented in Gray, John, Stuart, 
Lawrence, & Atwood, 1990.) 

In this illustration, performance time is not the only variable of interest in this system, and 
it is probably not the most important variable. In fact, the NTD actually pauses a full 12 
seconds before initiating the hail to FLT, much longer than the 2.5 seconds necessary to 
find the next point of participation. Further data collection (e.g. videotapes as well as audio 
tapes) is necessary to determine whether the NTD is filling the interval with other time- 
critical tasks, recording information for future use, looking further ahead, etc., or simply 
waiting for FLT to initiate the request himself before initiating the offer. Under the former 
conditions, the response time of a system may be important to allow the NTD the slack time 
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(Estimates for the duration of perceptual, cognitive, and eye-movement operators are taken 
from John, 1990. The estimate for the turn-page operator duration is from Egan, 1991.) 





unknown 




Figure 1 3. Processes required for the NTD to use a hypothetical computer support system to find the next step in 
which he participates, after completion of step 16-0907. 

(Estimates for the duration of perceptual, cognitive, and motor-movement operators are taken from John, 1990.) 
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to pursue the other tasks he is accustomed to perform in this timeframe. In the latter case, 
system response time would not be on the critical path for the larger procedure and would 
not be as important. In either case, it is a good rule of thumb to ensure that any new 
system not be worse in any measure than die current paper OMI, and this type of analysis 
is useful in evaluation of performance time for a proposed system. 

6.2. Organizational What-If Evaluations 

The NTD is an important individual in the coordination of a shuttle launch, and a 
computational model that can be used to evaluate computer support systems for his 
individual tasks, as described in section 6. 1 is a contribution to the design of those support 
systems. However, the NTD is only one member of large team where every member's 
performance has an impact on the launch. Any computer support system introduced into 
the launch procedure will most likely be used by many members, not simply to support the 
NTD's individual tasks. Therefore, it is appropriate to ask whether a computational model 
like NTD-Soar can help evaluate the impact of computer support aids on the performance of 
the entire team, rather than just the performance of the NTD alone. 

Research by a group of social scientists, organizational behavior researchers, and computer 
scientists, with another Soar system, called Plural-Soar, indicates that linking together 
several simulated intelligent agents and giving then a task to do collectively, can lead to the 
emergence of recognizable organizational behavior phenomena (Carley, Kjaer-Hansen, 
Prietula, and Newell, 1991). 

Plural-Soar's task was to fill a series of orders in a warehouse, and the researchers varied 
the number and capabilities of the agents. The number of agents varied from one to five. 
The least capable agent, the basic agent, could perform the order-filling task without any 
memory for the location of objects nor any communication capabilities with which to 
interact with the other agents. The most capable agent could remember where objects were 
located in the warehouse and broadcast questions to the other agents and receive answers as 
to where objects were located. The researchers then played what-if on an organizational 
level, discovering the effects of number of agents and capabilities on several 
organizationally interesting performance measures. These simple agents reproduced 
several well-known organizational results including that time to complete the task showed a 
non-linear decrease with the number of agents (decreasing returns to scale), cognitive and 
physical effort per agent showed a non-linear decrease with the number of agents (there 
was overhead involved with dealing with the other agents), and the waiting time increased 
with the number of agents (they got in each other's way). 

Although Plural-Soar's task and simulations of the individual agents were very simple, the 
results are indicative of the kind of organizational phenomena that can be investigated using 
that approach. NTD-Soar, with its access to external memory for the situation, that is, the 
marked-up copy of the OMI, and its ability to accept simple communications from launch 
team members ("step 16-0910 complete") is comparable to Plural-Soar's most capable 
agent. The researchers discuss several directions in which such an agent could be further 
specified to produce opportunities for more interesting social behavior to emerge. Among 
the directions discussed are more situation knowledge, learning from experience, and more 
sophisticated communication capabilities. All of these are on the agenda for NTD-Soar. 

6.3. Speculations about using discourse analysis in the design of computer support 
systems for the NTD 

The preliminary discourse analysis presented here has provided information about 
discourse situations, conversations, information exchanges, speech acts, OIS maxims of 
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cooperative communication and a goal hierarchy. However, models of other tasks with 
large conversational components have been successful in evaluating computer support 
systems without using such analysis (Gray, et. al., 1990); what value does this information 
add to a computational model of the NTD? 

To examine the value added by the discourse analysis, consider a computational model 
constructed without reference to the conversational behavior on the OIS. A minimally 
competent model could be constructed using the OMI alone (call it the OMI -NTD -model). 
The OMI-NTD-model would have a goal structure identical to that inferred from the 
discourse analysis, because that goal structure is also built into the format of the OMI: steps 
are grouped into sections, sections into sequences, with the ultimate goal being the launch 
itself. Operators for satisfying the goals in the OMI-NTD-model would be derived from 
the two-dimensional encoding of information on the written page (scanning operators) and 
the content of that information. That is, reading operators would match incoming 
information to specific locations in the OMI, generate messages, and initiate communication 
episodes in which the OMI-NTD-model plays a part Such a model would be sufficient for 
making the evaluation of the computer display system posited in section 6.1. 

The OMI-NTD-model, however, would not produce the conversation observed on the OIS. 
It would be able to generate a summons from the CMD and RESP columns of the OMI, but 
it would have no knowledge that a conversation should occur; it would proceed from the 
summons to the message (e.g., a literal reading of the DESCRIPTION of a step) without 
waiting for an answer to the summons or a registration of the message. It would be a 
model of communication that assumes all messages are heard and acted upon as soon as 
they are spoken on the OIS. It would be a model of the NTD in isolation receiving auditory 
information from, and delivering verbal messages to, an amorphous external entity. 

The preliminary discourse analysis produced a much richer model of communication than 
examination of the OMI alone. The OIS maxims provide knowledge that produces 
complete conversations. They provide knowledge that allows for variation in message 
content (e.g., maxim l.a.i allows either the SEQ or DESCRIPTION to be used in 
identifying the step targeted for completion) and could produce variable message content 
constrained by the maxims with a simple random process for selection between alternative 
for ms Such a model, indeed even its current implementation in NTD-Soar, produces a 
communication flow much more easily identified with the actual behavior. This level of 
model, however, cannot produce the violations of the OIS maxims observed in the 
communications. The information exchanges presented here, with the pre-and post- 
conditions of the component speech acts, can help understand such variability when it is 
observed, but they cannot generate that variability. An even richer model of human 
communication is required for that level of prediction. 

The discourse analysis, then, provides additional operators and preferences for operator in 
our Soar model, beyond what an analysis of the OMI could provide. What role does this 
richness play in the computational model as a design tool? As stated above, the minimal 
OMI-NTD-model could be used for the individual what-if analysis of Section 6.1; the 
discourse information adds nothing to the prediction of the NTD's perceptual, cognitive, 
and motor processes not directly involved in communication. 

The discourse analysis does, however, give information about the assumptions behind 
communications, which is potentially useful in qualitative evaluation of computer support 
systems. Every computer support system will have implications for the assumptions of 
communication described in the OIS maxims and speech acts, and these implications can be 
weighed against the current technology. For instance, consider a sophisticated computer 
support system that combined on-line displays of OMI information with limited speech 
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recognition that could pick out the calls-signs of launch team members. (A speech 
recognition system with limited vocabulary and limited users, with an abundance of 
training materials in the foim of previous launch tapes, is not beyond current technology.) 
Imagine an interface that allowed a subsystem manager to point to the OMI step he or she 
wished to report complete to the NTD. The subsystem manager would hail the NTD, the 
support system would recognize when the NTD answered the summons, and then 
automatically display the indicated step on the NTD's terminal. Such a system would 
change the OIS maxims in several ways. For instance, the step number or description 
would no longer be required verbally (maxim Lari) and conversations may reduce to a 
summons and answer, followed by the assertion "Complete" and the NTD's registration, 
"Copy". This interface would also effect the pre- and post- conditions of the speech acts 
and change the logical chain through information exchanges. The incomplete conversation 
discussed in Section 4, which was missing only the answer to the summons, could no 
longer be understood because the computer support system could not bring the NTD to the 
correct location in the OMI without that answer. We would predict that a summons would 
always be answered with this system, or the summoner would issue the summons again 
before proceeding. 

This analysis shows differences between discourse situations derived from the technology 
involved. It does not, however, provide any quantitative measures with which to judge the 
value of those differences. It also remains to be determined how many such differences 
could be derived solely from an analysis of the technology involved, without reference to 
actual behavior with that technology. In addition, the discourse analysis does not give 
much guidance as to how to design new support systems. (Examining the assumptions in 
the current OIS maxims and speech acts gives some indication of what might be varied, but 
no guidance as to how to vary it with technology or in which direction to go.) Much work 
must be done to bring discourse analysis into the designer’s toolbox, making it quantitative 
and prescriptive, but preliminary indications are that it is a promising path to explore. 

7. CONCLUSIONS 

We have presented analyses of the job of the NTD and a computational model of the NTD 
performing that job. The computational model seems adequate for quantitative evaluation of 
the performance time of individual, routine cognitive tasks that are a part of the total task of 
the NTD. The discourse analysis provides information that can be incorporated into the 
model to produce fairly rich simulated communications, and can be used to reason 
qualitatively about the effects of computer support systems on the communication patterns. 
Although potentially useful as it stands, this model touches only the routine, baseline 
behavior of the NTD; it is still to be extended to include the critical aspects of situation 
assessment, orientation in anomalous situations, training and learning, and the interaction 
of computer support systems with these vital behaviors. 
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Appendix 2. Speech Acts in OIS Communications 



KEY TO THE FORMAT OF THIS APPENDIX 

Speech Act - FORM = ATI. CAPITALS indicates the contents of columns in the OMI, 

"Quotes" indicate specific words that are commonly used 
[square brackets] indicate an optional part of the speech act 
I indicates an "or" relation with the adjacent term 
PRECONDITIONS are what the speaker believes to be true in the world 
prior to the speech act 
[a list of the speaker's beliefs follows] 

POSTCONDITIONS are what is true in the world after the speech act 
[a list of the truths follows] 


Summons - FORM = RESP, CMD, [Channel] 

PRECONDITIONS 

1. the summoned party is passively listening 

2. the summoned party will actively listen to the communication after 
hearing the summons 

3. the summoned party can answer the summons 

4. the summoned party will answer the summons when the summoning 
party stops talking 

5. the summoned party can know who the summoning party is from the 
content of the summons 

6. the summoning party knows who the summoned party is 

POSTCONDITIONS 

1 . the summoning party is actively listening for an answer 

2. the summoning party is expecting an answer now 

3. the summoned party has had an opportunity to hear the summons 

4. the summoned party has had an opportunity to know who the 
summoner is 


Answer - FORM = RESP, ["Go” I "Go ahead"l] [CMD] 

PRECONDITIONS 

1. the summoned party has heard the summons 

2. the summoned party knows who the summoning party is 

3. the summoning party is expecting an answer now 

4. the summoning party is actively listening for an answer 
POSTCONDITIONS 

1. the summoned party is actively listening 

2. the summoned party knows who the summoning party is 

3. the summoned party is waiting for further information from the 
summoning party 

4. the summoning party has had an opportunity to hear the answer 


Request - 


Promise - 


FORM = assertive I directive 

PRECONDITION S 

1 . the requestor is at the correct OMI location 

2. the requestor is authorized to make the request, as written in the OMI 

3. the requestor knows who the requestee is 

4. the requestee is the correct person to receive the request, as written in 
the OMI 

5. the requestee is actively listening ----- -- - 

6. the requestee can understand the request 

7. the requestee can get to the correct OMI location via SEQ or 
DESCRIPTION 

8. the requestee has the information, or the ability to get the information, 
needed to comply with the request 

9. the requestee knows who the requestor is 

10. the requestee will give the information to the requestor when it is 
available 

POSTCONDITIONS 

1. the requestor is actively listening 

2. the requestor is expecting a compliance or promise now 

3. the requestor wants the information now 

4. the requestor will be satisfied with a promise to provide the 
information in the future 

5. the requestee has had an opportunity to hear the request 


FORM = commissive 
PRECONDITIONS 

1. the requestee has heard the request 

2. the requestee understands the request 

2. the requestor is authorized to make the request, as written in the OMI 

3. the requestee knows who the requestor is 

4. the requestee is the correct person to receive the request, as written in 
the OMI 

3. the requestee does not have the necessary information now 

4. the requestee can get the necessary infor matio n later 

5. the requestor is actively listening 

6. the requestor is already at the request in the OMI 

7. the requestor wants the information now 

8. the requestor will be satisfied with a promise to provide the 
information later 

POSTCONDITIONS 

1. the requestee is committed to comply with the request in the future 

2. the requestee is actively listening 

3. the requestee expects a registration of the promise now 

4. the requestor has had the opportunity to hear the promise 


Registration - 


Compliance - 


FORM = "Copy" I "In work" I "OK" I others 
PRECONDITIONS 

1 . the partner is actively listening 

2. the partner is expecting a registration 

3. the registering party has heard the information 

4. the registering party has understood the information 

5. the registering party is at the step location in the OMI (or will go to it 
as needed) 

6. the registering party is authorized to do the action implied by the 
information 

6. the registering party is able to do the action implied by the information 
for a promise, there is nothing to do at this time 
for a compliance, this involves marking down the information in the 
OMI as appropriate 

for a direction, this involves doing whatever was directed 
for an assertion, this involves marking down the information in the 
OMI as appropriate 

for a refusal, this involves marking down the information in the OMI 
as appropriate 
POSTCONDITIONS 

1. the registering party is committed to doing the action implied by the 
information 

2. the partner has had the opportunity to hear the registration 


FORM = assertive 
PRECONDITIONS 

1. the requestee has heard the request 

2. the requestee understands the request 

2. the requestor is authorized to make the request, as written in the OMI 

3. the requestee knows who the requestor is 

4. the requestee is the correct person to receive the request, as written in 
the OMI 

5. the requestee is at the correct location in the OMI 

6. the requestee has the necessary information 

7. the requestor is actively listening 

8. the requestor is already at the request in the OMI 

9. the requestor wants the information now 
POSTCONDITIONS 

1. the requested information was supplied 

2. the requestee is actively listening 

3. the requestee expects a registration of the compliance now 

4. the requestor has had the opportunity to hear the compliance 


Direction - 


Assertion - 


Offer - 


FORM = directive (often exact words from the OMI) 

PRECONDITIONS 

1 . the director is at the correct OMI location 

2. the director is authorized to make the direction, as written in the OMI 

3. the director knows who the directee is 

4. the directee is the correct person to receive the direction, as written in 
the OMI 

5. the directee is actively listening 

6. the directee can understand the request 

7. the directee can get to the correct OMI location via SEQ or 
DESCRIPTION 

8. the directee is capable of performing the direction 

9. the directee knows who the director is 
POSTCONDITIONS 

1. the director is actively listening 

2. the director is expecting a registration now 

3. the directee has had an opportunity to hear the direction 

FORM = assertive 
PRECONDITIONS 

1. the partner is actively listening 

2. the partner can locate the step in the OMI via SEQ or DESCRIPTION 

3. the asserter has the authority to make the assertion, as written in the 
OMI 

4. the asserter knows who the partner is 

5. the partner is the proper person to receive die assertion, as written in 
the OMI 

6. the partner knows who the asserter is 
POSTCONDITIONS 

1. the asserter is actively listening 

2. the asserter is expecting a registration now 

3. the partner has had an opportunity to hear the assertion 

FORM = assertive, directive 
PRECONDITIONS 

1 . the offeror is at the correct OMI location 

2. the offeror is authorized to make the offer, as written in the OMI 

3. the offeror knows who the offeree is 

4. the offeree is the correct person to receive the offer, as written in the 
OMI 

5. the offeree is actively listening 

6. the offeree can understand the offer 

7. the offeree can get to the correct OMI location via SEQ or 
DESCRIPTION 

8. the offeree knows who the offeror is 

9. the offerer has the information offered at the time of the offer 

10. the offerer is willing to give the information to the partner at the time 
of the offer if the partner accepts 

1 1. the offerer is willing for the offeree to refuse (otherwise it would be a 
direction, not an offer) 

POSTCONDITIONS 

1. the offeror is actively listening 

2. the offeror is expecting an acceptance or refusal now 

3. the offeree has had an opportunity to hear the offer 



Acceptance - 


Refusal - 


FORM = assertive, directive 
PRECONDITIONS 

1. the offeree has heard the offer 

2. the offeree understands the offer 

3. the offeror is authorized to make the offer, as written in the OMI 

4. the offeree knows who the offeror is 

5. the offeree is the correct person to receive the offer, as written in the 
OMI 

6. the offeree does not have the necessary information now 

7. the offeree wants the information now 

8. the offeree has found the correct step in the OMI 

9. the offerer will give the information now after the acceptance is 
performed 

POSTCONDITIONS 

1. the offeree is actively listening 

2. the offeree expects the information offered now 

3. the offeror has had the opportunity to hear the acceptance 

FORM = assertive 
PRECONDITIONS 

1. the offeree has heard the offer 

2. the offeree understands the offer 

3. the offeror is authorized to make the offer, as written in the OMI 

4. the offeree knows who the offeror is 

5. the offeree is the correct person to receive the offer, as written in the 
OMI 

6. the offeree does not want the information now 

7. the offeree has found the correct step in the OMI 
POSTCONDITIONS 

1. the offeree is actively listening 

2. the offeree expects a registration of the refusal now 

3. the offeror has had the opportunity to hear the refusal 




